🕷️️ Job Radar • SCRAPING

freelancer.com 🟡 2026-05-28

🔹 AI-Driven Procurement Intelligence & Web Crawling Platform
👤 Client: 🇮🇳 Nagpur, India Member since 2024-06-13
💰 Price: $276 Average bid
🚩 Problem: Lack of a centralized, automated system to extract and structure procurement data from fragmented public and corporate portals.
📦 Existing: Not specified

Specifications:

[Target] Government, eProcurement, PSU, Railway, Municipal, Smart City, and Corporate portals
[Method] Scheduled incremental crawling, dynamic content handling, pagination, session/token management, anti-block/proxy rotation, queue-based processing
[Method] PDF parsing, OCR (Tesseract, PaddleOCR), AI-assisted metadata extraction, document classification
[Method] AI workflows for keyword classification, duplicate detection, summarization, and context-based extraction
[UI/UX] Responsive dashboard (Desktop, Tablet, Mobile) with analytics widgets, advanced filters (State, Category, Department), and real-time crawl status
[Stack] Python (FastAPI/Django/Flask), Scrapy, Playwright, Selenium, BeautifulSoup
[Stack] OpenAI API, Anthropic Claude API
[Stack] PostgreSQL, Redis, Celery, MinIO (S3-compatible)
[Stack] React.js, Docker, AWS (EC2, S3)
[Stack] Grafana for operational monitoring
[Security] Role-based access control (RBAC), secure authentication, API rate limiting, credential encryption
[Format] REST API, Structured JSON metadata, PDF/OCR documents

Workflow:

1. Automated crawling of target procurement portals via distributed queue.
2. Extraction of raw metadata and document retrieval.
3. OCR and AI-driven parsing of PDFs for structured data intelligence.
4. Data normalization and storage in PostgreSQL/MinIO.
5. Visualization of insights and alerts via responsive frontend dashboard.
6. Notification dispatch via Email and WhatsApp.